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DETAILED ACTION 

1 . This communication is responsive to the Amendment filed 04 August 2005. 

2. Claims 23-42 are pending in this application. Claims 23, 41 and 42 are 
independent claims. In the Amendment filed 04 August 2005, claims 23, 25, 26, 28, 41 
and 42 were amended. This action is made Final. 

Claim Rejections - 35 (JSC § 103 

The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

3. Claims 23-31 and 37-42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent No. 5,894,306 to Ichimura in view of U.S. Patent 
Application Publication No. 2002/0085759 to Davies et al. and U.S. Patent No. 
6,363,380 to Dimitrova, and further in view of U.S. Patent No. 5,761 ,340 to Suzuki. 

Referring to claim 23, Ichimura discloses a system and method for collecting and 
archiving multimedia information substantially as claimed. See Figures 1-12 and the 
corresponding portions of the specification for this disclosure. Ichimura discloses a 
multimedia information collection control apparatus, comprising: 

a multimedia information collection unit [Electronic Meeting Device 10] configured 
to collect multimedia information ['image data', 'audio data' and 'user input from 
electronic pen 13'] from a plurality of kinds of input devices [Video Camera 16 (image), 
Microphone 15 (audio), and Touch Panel/Tablet 12 (pen input)]; 
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a multimedia correspondence memory [First Memory Section 22] configured to 
correspondingly store the multimedia information including an image and a speech [The 
audio signal from microphone 15 and the image signal from video camera 16 are 
sequentially stored into the first memory section 22' (Column 8, line 62 et seq.)]; 

a display [Screen 11] configured to output the image [See column 7, lines 13-20] 
and a window including a plurality of characters [See Column 7, lines 48-51]; 

an information recognition unit [State Detector 24] configured to recognize 
characters in the image [See Column 7, lines 48-51], to extract data from the 
recognition result of the characters [See Column 7, lines 48-51 , and to identify a 
speaker from the speech ['a speaker is recognized according to the extraction of 
features of the input audio signals' (Column 8, lines 13-20) and 'the method used to 
I identify a speaker... can involve identification based on the facial and mouth motions of 

I the speakers determined from the image data' (Column 15, lines 5-20)]; and 

a second memory section [23] configured to store relational pointers to the 
multimedia information [stored in the first memory section] in correspondence with the 
identified speaker by using the personal data ['the name of the recognized speaker is 
transferred to the second memory section 21 as a result of detection' (Column 8, lines 
1 3-20) also see Figures 5-12 and the corresponding portions of the specification for the 
disclosure of the analysis]. 

Ichimura does not explicitly extract personal data from the recognition result of 
the characters as claimed. However, it is clear that personal data handwritten on the 
tablet (12) would be recognized by the character recognition unit just as any other 
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handwritten text would be. Thus, one could easily infer that personal data could be 
extracted from its character recognition. Further, Davies discloses a system and 
method similar to that of Ichimura, wherein personal data (e.g. a person's name) is 
extracted from optical character recognition of an image (e.g. of a business card) to 
identify a person, and then data is stored in correspondence with the person's name 
within a database. See paragraph 0014 for the details of this disclosure. 

Ichimura does not explicitly teach that the display is configured to output "an 
attribute selection window including a plurality of attribute items of characters"; the 
information recognition unit has "a plurality of knowledge dictionaries each 
corresponding to each of the plurality of attribute items"; selecting an attribute item from 
the plurality of attribute items in response to a user's indication; and "using one of the 
plurality of knowledge dictionaries corresponding to the one attribute item" as claimed. 
However, Suzuki and Davies teach: the display presents an attribute selection window 
[Suzuki: See Figs. 7-1 1 ; Davies: See Figs. 4-5] including a plurality of attribute items 
[Suzuki: edit commands; Davies: 'stickers']; a plurality of knowledge dictionaries 
[Suzuki: functions defining actions of edit commands; Davies: services (See 414 & 516- 
518)] each corresponding to each of the plurality of attribute items [Suzuki: one action 
function per edit command; Davies: a specific service is associated with each 'sticker' 
(attribute)]; selecting one attribute item from the plurality of attribute items in response to 
a user's indication [Suzuki: See Figs. 7-11; Davies: user applies a specific 'sticker']; and 
recognizing characters in the image by using one of the plurality of knowledge 
dictionaries corresponding to the one attribute item [Davies: action corresponding to the 
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service is executed; Suzuki: action function is executed corresponding to the editing 
command selected] as claimed. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate Davies' functionality of extracting of a person's name 
by optical character recognition into the system and method of Ichimura, and further to 
incorporate the attribute selection display and knowledge dictionary functionality of 
Davies and Suzuki into Ichimura's system and method, to allow the user to select an 
attribute for recognition using a corresponding knowledge dictionary. One would have 
been motivated to do so in order to identify and store information corresponding to a 
new user of the system without necessity of manual input, as described by Davies, and 
further to provide the user a more flexible interface with increased functionality, as 
described by both Davies and Suzuki. 

Neither Ichimura, Suzuki nor Davies explicitly discloses "a multimedia database" 
configured to relationally store the multimedia information in correspondence with the 
identified speaker as claimed. However, Ichimura's second memory section 23 does 
store this relationship as described above and shown in Figure 12. Dimitrova discloses 
a system and method for gathering multimedia data and storing it with meaningful 
relationships between the different types of data very similar to that of Ichimura. See 
Figures 1-6B and the corresponding portions of the specification for this disclosure. 
Dimitrova's event relationships are stored as 'video stories' in a video story database 
(multimedia database). Refer specifically to column 6, line 60 - column 7, line 55 and 
column 1 0, line 20 - column 1 3, line 1 7 for the details of this disclosure. Thus, 
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Dimitrova teaches a multimedia database configured to relationally store the multimedia 
information as story data analyzed by the system. 

Given the desire for increased memory space to store multimedia data for 
meetings that last an extensive amount of time, the desire for access to a larger archive 
of multimedia data for many meetings, and the equivalent functionality of the systems, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to swap Ichimura's (as modified by Suzuki and Davies) Second Memory Section 
23 with a multimedia database, such as that of Dimitrova, to store the multimedia 
information (image data, audio data, and handwritten user input data) relationally in 
correspondence with the person's name analyzed by Ichimura's state detector 24. 

Referring to claim 24, the combination of Ichimura, Dimitrova, Davies and Suzuki 
as applied to claim 23 above (hereafter 'Ichimura/Davies/Dimitrova/Suzuki') teaches the 
multimedia information collection control apparatus of claim 23, as above, wherein the 
plurality of kinds of input devices include a camera to input the image [Ichimura: 16], 
and a microphone to input the speech [Ichimura: 15] as claimed. 

Referring to claim 25, Ichimura/Davies/Dimitrova/Suzuki as applied to claim 23 
above teaches the apparatus of claim 24, as above, wherein said multimedia 
information collection unit [Ichimura: Electronic Meeting Device 10] includes an indicator 
\ [Ichimura: Electronic Pen 13 in conjunction with Touch Panel/T ablet 12; Suzuki: See 

Figs. 1-9] to artificially specify a recognition area [Suzuki: See Figs. 1 , 3A-3D, 5, 7A-7D 
& 9 and column 5, lines 29-33] of the image on said display [Ichimura: see above]. 
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Referring to claim 26, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 25, as above, wherein the plurality of attribute items [Suzuki: edit commands; 
Davies: 'stickers'] include a data, a meeting name...[lchimura: See Figs. 7-12; Davies: 
See above] as claimed. 

Referring to claim 27, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 26, as above, wherein said indicator artificially selects one attribute [Suzuki: 
based on the mark] corresponding to the characters in the image [Davies: sticker (See 
414 & 516-518)].. .as claimed. 

Referring to claims 28 and 29, Ichimura/Davies/Dimitrova/Suzuki teaches the 
apparatus of claim 27, as above, wherein if the one attribute item is a card, said 
information recognition unit recognizes the characters in the image by using one 
knowledge dictionary [See Davies' place in combination as in claims 23 & 26-27 above] 
corresponding to the card [See above]; wherein if the one attribute item is a card 
[Davies: business card (See paragraph 0014)], said multimedia database stores the 
recognition result of the characters of the card as the speaker's personal data [See 
claim 23 above] as claimed. 

Referring to claim 30, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 29, as above, wherein said information recognition unit [Ichimura: state detector 
24] includes a speech recognition unit [Speaker Recognition from audio data (See 
Figures 5 & 6 and the corresponding portions of the specification)] configured to 
recognize the speech in the multimedia information; and wherein said multimedia 
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database [See claim 23 above] stores the recognition result of the speech [Ichimura: 
See Figure 12 & claim 23 above]... as claimed. 

Referring to claim 31 , Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 29, as above, wherein said information recognition unit [Ichimura: state detector 
24] includes a face recognition unit [Speaker Recognition from image data (See Column 
15, lines 5-20)] configured to recognize a facial characteristic of a face area in the 
image; and wherein said multimedia database [See claim 23 above] stores the 
recognition result of the facial characteristic [Ichimura: See Figure 12 & claim 23 
above]... as claimed. 

Referring to claim 37, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 23, as above, wherein said multimedia database correspondingly stores [See 
claim 23 above] a record identification [Ichimura: Participant ID - user input ID] of each 
item of the multimedia information as claimed. 

Referring to claim 38, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 37, as above, wherein said multimedia information collection unit [Ichimura: 
See Figs. 1-2] includes an information addition unit [Ichimura: 10] configured to 
additionally input information to said multimedia database [See claim 23 above]; and 
wherein said multimedia database additionally stores the input information [See claim 
23 above]... as claimed. 

Referring to claim 39, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 37, as above, further comprising a dialogue control unit [Ichimura: Playback 
Specifying Section 27 (column 12, line 57 - column 14, line 42)] configured to input a 
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retrieval request from a user [(See column 13, lines 8-20)], to analyze the retrieval 
request [(See column 13, lines 26-37)], and to generate a retrieval key ['the X-Y 
coordinates groups' (Again, see column 13, lines 26-37)] according to the analysis 
result as claimed. 

Referring to claim 40, Ichimura/Davies/Dimitrova/Suzuki teaches the apparatus 
of claim 39, as above, further comprising a retrieval control unit [Ichimura: Control 
Section 30 in conjunction with Playback Specifying Section 27] configured to retrieve 
the multimedia database by comparing the retrieval key with the record identification of 
each item of the multimedia information [(See column 13, lines 30-48)]; and wherein 
said multimedia information collection unit [Electronic Meeting Device 10] presents the 
retrieved information through the display [playback of image data (See Figure 2)] as 
claimed. 

Claims 41 and 42 are rejected on the same basis as claim 23. See the 
discussion regarding claim 23 above for the details of this disclosure. 

Allowable Subject Matter 

4. Claims 32-36 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

5. The following is a statement of reasons for the indication of allowable subject 
matter: 
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Ichimura (U.S. 5,894,306), Dimitrova (U.S. 6,363,380), Davies (US 
2002/0085759) and Suzuki (U.S. 5,761,340) each fail to teach or suggest "wherein said 
indicator artificially specifies the recognition area of the image on said display by using a 
mark; and wherein a shape of the mark corresponds to a kind of a recognition object" as 
recited in claim 32. 

The remaining prior art of record also fails to teach or suggest this distinguishing 
limitation, and therefore fails to anticipate or make obvious the systems for multimedia 
information collection control of claims 33-36. 

Response to Arguments 

6. Applicants' arguments filed 04 August 2005 have been fully considered but they 
are not persuasive. 

Referring to applicants' remarks on pages 12-14 regarding the Section 103 
rejections of claims 25-26, which subject matter therefrom has been added to the 
independent claims by the Amendment of 04 August 2005: Applicants argued that the 
various references do not teach or suggest each and every element of the claims. 
The examiner disagrees for the following reasons: 
Applicants' arguments constitute a piecemeal analysis of the references 
individually, failing to consider the combination as a whole and what would have been 
obvious to one of ordinary skill in the art at the time of invention. In response to 
applicant's arguments against the references individually, one cannot show 
nonobviousness by attacking references individually where the rejections are based on 
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combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 
1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). First, 
contrary to applicants' assertions on pages 11-12, the examiner did not concede that 
Ichimura and Dimitrova fail to teach a display. Ichimura clearly teaches "a display 
configured to output the image" as claimed. Second, Davies does not teach away from 
an attribute selection window, contrary to applicants' assertions. A user of Davies' 
system must still apply a sticker. This corresponds to attribute selection. That Davies' 
paragraph 0017 "does not require the user to enter commands" does not constitute a 
teaching away, and is irrelevant to the combination as a whole. Finally, applicants' 
assertion that Davies does not teach the claimed "knowledge dictionaries" is 
unsubstantiated. Applicants have provided no evidence or reasoning as to why the 
claimed "knowledge dictionaries" are different from Davies' services. The examiner 
interprets the claimed "knowledge dictionaries" to be functions corresponding to the 
attributes. Due to the lack of concrete definition of "knowledge dictionary" in applicants' 
specification, this interpretation is certainly reasonable. Thus, Davies services are 
"knowledge dictionaries" as claimed. 

Referring to applicants' remarks on pages 14-15 regarding the Section 103 
rejections of claims 25-26, which subject matter therefrom has been added to the 
independent claims by the Amendment of 04 August 2005: Applicants argued that 
there is no motivation to combine the references. 

The examiner disagrees for the following reasons: 
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Applicants' arguments [i.e. "The Examiner failed to provide any evidence of 
motivation to combine the cited references (Office Action at p. 1 1 referring to claim 26)"] 
appear to be based on a supposed lack of motivation directed specifically to claim 26. 
However, claim 26 depends from claim 25. In the previous Office Action, clear and 
explicit motivation to combine the references was provided in regard to the 
combinations set forth regarding claims 23 and 25. The combination of references set 
forth regarding claim 25 disclosed the features of claim 26. Thus, the motivation to 
combine the references to obtain the invention of claim 26 was the same as that set 
forth with regard to claim 25. In response to applicant's argument that there is no 
suggestion to combine the references, the examiner recognizes that obviousness can 
only be established by combining or modifying the teachings of the prior art to produce 
the claimed invention where there is some teaching, suggestion, or motivation to do so 
found either in the references themselves or in the knowledge generally available to one 
of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 
1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, 
motivation to combine comes from both the references themselves and from the 
knowledge generally available to those of ordinary skill in the art, as specifically shown 
in the grounds for rejection of claim 23 above. 

Conclusion 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Goddard whose telephone number is 571-272- 
4020. The examiner can normally be reached on M-F, 9 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on 571 -272-4023. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



bdg 

6 October 2005 
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